‘Transformation: a report 


Stephen Whitmarsh!” 


‘Institute of Brain and Spinal Cord (ICM), Pitié-Salpétriére Hospital, Paris, France 
7141=3 


June 2019 


1 Introduction 


”The growth of musical art in any age is determined by the technological progress which parallels it. (...) 
If we admit that the creative imagination of the composer may form musical ideas which, under the specific 
conditions of a given epoch, cannot be translated into sounds, we acknowledge a great dependence of the 
artist upon the technical position of his era, for music attains reality only through the process of sound.” 
(Schillinger, 1931) 


2 Thursday May 30th: R1 Reaktorhallen 


We began our events on the evening of May 30th 
at the decomissioned nuclear reactor hall R1, 25 
memters under the centre of Stockholm. I pre- 
sented a basic introduction, starting from Hans 
Berger’s discovery of human EEG and alpha activ- 
ity (Berger, 1931), to the first sonification for scien- 
tific purposes (Adrian & Matthews, 1934), the first 
EEG-sonification in an artistic performance (Lucier, 
1982), to our own open-source initiative develop- 
ing open-source software (Oostenveld & Whitmarsh, 
2015) and its use for real-time EEG analysis in 
artistic performances (Whitmarsh, Huttner, Huhta, 
Takahashi, & Oostenveld, 2016). We ended with the 
ubiquitous presence of feedback systems in nature, ; LA 
and between (human) beings in particular. Con- ROBERT OOSTENVELD “So | „SAMON TAKAHASHI 
sider, for example, a mother suckling a baby: The - : z 
suckling of a baby stimulates the nerves in the nip- 

ple, which result in the hypothalamus increasing the 
release of oxytocin and prolactin in the pituitary 
gland, leading to an increased lactation which in 
turn increases the suckling behavior of the baby, l 
until the baby is satiated. Looking at feedback BIOFEEDBACK ana BRAINWAVE MUSIC 
in brain-computer(-music)-interfaces in this way de- eoo oloo eooo ET ald 
emphasizes the digital and ‘cold’ aspect of such as inborn 3 : 
system, and emphasizes the emergence of new cy- 

bernetic systems in which causality and control are 

circular and emergent, rather than linear and indi- Figure 1: Event poster by Samon Takahashi 
vidual. This emphasis on emergence rather than on 


control has evolved from our practice over the years. We came to realize that the attempt to consciously 
control music with EEG is, in fact, not the most interesting or satisfying aspect for the performer, and in 
fact precludes full immersion into a brain-music connection. We found that a more receptive and passive 
attitude often resulted in a more profoundly relaxing and psychedelic! experience. Secondly, measurements 
of EEG are always noisy and the relationship between EEG signals and mental functions at best ‘loose’ as 
well. Since perfect fidelity and perfect conscious control of EEG are impossible, therefor, an unavoidable 
unpredictability is brought into the system, which can be either ignored or used constructively. It is quite 
uncommon to see these issues explored in an honest and creative way. David Rosenboom, however, has 
explored unpredictability in music and the ramifications of brain-music feedback loops over the course of 
his career. The presentation finished, therefor, with a quote from his Biofeedback and the arts, results of 
early experiments (Rosenboom, 1976): “/Biofeedback in art] should be seen as the circulation of information 
about functions within an organism in ever widening feedback loops involving the consciousness of that or- 
ganism, to serve its creative extension, beyond the structure of its own prior self-definition, in the natural 
meta-evolution of its self-organization dynamics” . 

It was then time for a series of performances by Atau Tanaka: Le Loup, Lifting and Myogram. Atau first 
took a couple of minutes to introduce the two multichannel EMG arm bracelets, which he would be using to 
control different aspects of sound. I was impressed how clearly he was able to explain the techniques behind 
what would be happening, in only a couple of minutes and without the use of a single visualization (in 
contrast with my scientist’s PowerPoint presentation). However, it did not prepare for the magnitude of the 
performance. Besides the responsiveness of his system, his muscles and movements mapped also perfectly to 
the three-dimensional sound diffusion, amplifying his body into a musical instrument as large as the room. 
His fingers and arms moved sometimes small and jazzy, sometimes expansive and dramatic, as a conductor 
of an orchestra of invisible sonic ghosts. I was able to discern the use of EMG envelopes, as well as the 
sonification of ’raw’ muscle activity, the latter sounding like electricity which he conjured up into almost 
tangible shapes around his body, throwing them around the audience in loud sweeps that made the rigging 
of the reactor hall charm with resonating overtones. It was an arresting, visceral experience. 

It was then time for a break and a visit to the bar which Per set up, in wonderful kitschy colors contrasting 
with the grayness of the reactor hall. During dinner, only an hour before, we discussed how much we should 
charge for a beer. Fifteen minutes later, Robert had Per Huttner (who was serving as the bar man) wear 
a chest belt, and programmed the EEGsynth in such a way that a screen on the bar displayed the price of 
beer depended on his heart-rate. It was quirky and fun, and the audience accepted the fluctuating prices 
with enthusiasm. 

The last performance of the day was by David 
Rosenboom, who started and ended with two solo 
pieces on piano (including resonators and electronic 
synthesis), bookending his third and newest real- 
ization of Portable Gold and Philosophers Stones, 
first created in 1972 (Rosenboom, 1975), just then 
re-released with the suffix: Deviant Resonances 
(Rosenboom, 2019). Where Atau’s performance still 
provided a way to see the relationship between the 
biological signal (EMG) and its sonification by his 
use of gestural movements, we now dealt with the 
sonification of (EEG) brain activity provided by two 
audience members silently sitting and listening at 
the center of the stage. David himself played the 
piano and his violin that was connected to a mod- 
ular synthesizer using a 6 band fixed filter, enve- 
lope followers, VCAs and Make Noise’s Morphagene 
Figure 2: David Rosenboom performing with piano, tape loop module. EEG-based performances can 
violin and two EEG participants at R1 easily become be very static due to the immobility 

of whom the EEG is recorded. David’s performance, 


TAs in “Mind manifesting” from the English psyche + Greek délos ‘clear, manifest’ + -ic. 


however, moved wonderfully between his dynamic and energetic playing of his violin and the two absorbed 
active imaginative listener-performers (Rosenboom, 2014), in front of a slowly rotating video projection. The 
whole was transporting and absorbing. I tried to to pick apart the clusters of sound that swelled through 
the space, as David had introduced his piece in some detail and provided us some clues as what to listen 
for. Firstly, by responding to changes in the amplitude of alpha, musical events were triggered by changes 
in the attentional state of the participants. Furthermore, when two participants showed shifts of attention 
at the same time - e.g. in response to surprising sound in the music or the environment - resonating bells 
were heard. Afterwards, a question from the audience had the participants share some of their experiences, 
who reported being very deeply absorbed, loosing connection with time and space during the performance. 
The audience reported feeling themselves involved in the piece as well; an effect perhaps of the way David 
invited everyone to use their active imagination while listening to the music. 


3 Friday May 31st: Trädgården 


The next day we moved to Trädgården, the largest club in Stockholm, where we installed ourselved in their 
‘Red Room’ 

In the first presentation, Severine Samson addressed the question of why music is important to study the 
brain, and vice-versa, what neuroscience can tell us about how the brain works. Neuroscience of music is, in 
fact, a very young field and, as is the onus of science, has started by first deconstructing the phenomenon, 
clarifying its terms and processes. While arguably still dealing with very basic questions of music appreciation 
and composition, some remarkable insights can already be glimpsed. While not necessary for survival or 
satisfaction of primary biological needs, music appreciation is shown to be universal across times and cultures. 
Neuroimaging shows that the pleasure music evokes involves the reward circuit in much the same way as less 
healthy activities, such as gambling and drug addiction. In fact, the brain processes music as a sequence of 
predictive and reward cues that occur over time, leading to a constant sense of anticipation. It is this creation 
of expectations through its temporal structure, that provides us with pleasure. Severine then talked about 
how music can also be used to study emotion, shared experiences, memory, synchronized movement, and 
many other functions and behaviors for which, while topics in their own right, music can provide a natural 
paradigm for its manipulation and scientific study. A topic turned out to be particularly evocative was the 
universal response to physical characteristics of sound used in music, such as dissonance and harmony. What 
Severine’s research showed, was that the brain processes these ‘proto-musical’ stimuli very early on, within 
tens of milliseconds, after which higher-order brain regions interpret them and assign value and meaning to 
them. Like many other brain processes, the first responses of the brain represents the physical characteristics 
of the stimuli, while only later (but very quickly) our training, conditioning, attention and culture kicks in. 
Why dissonant stimuli seem to be universally regarded as negative by these early brain regions remains 
unknown, however. Art and science need each other if we want to truly get to the heart of it. For example, 
a methodological problem that researchers like Severine are struggling with, is the lack of musical stimuli 
that are novel and can span the emotional spectrum while remaining relatively similar in their physical 
characteristics. Currently, these stimuli are dominated by culture, such as the use of western film-music, or 
pieces of classical western music. Another animated converstion was triggered by her questions on how we 
should best understand improvisation. We discussed how improvisation and composition might represent a 
false distinction, a topic we will come back to later. Severine also voiced her enthousiasm in the promise of 
music-neurofeedback in learning and rehabilitation (the reward circuit!). If there was one thing that Severine 
was able to bring across, it was that all of us: the artist, the musician and the scientist, have unique and 
valuable knowledge to bring to the table when it comes to the (neuro)science of music, a topic which is by 
its very nature highly interdisciplinary. 

Samon Takahasi then presented his ongoing work in the history of brainwave music”. It is a completist 
project in which he keeps uncovering artistic works that have either used or stimulated the use of EEG 
recordings in visual art, music and performance. In fact, by now a full chronology would have taken much 
more time than we could possibly program. Furthermore, there are many stories to tell. As came up in our 
conversations several times, there is never a single beginning or a ’first’, that cannot be traced back to multiple 
pasts. Samon therefor gave a brief overview demonstrating the interwoven history of early experimental 


2a term actually borrowed from the title of David Rosenboom’s first EEG-based music record(Rosenboom, 1975) 


art, music by science, and the emergence of the artistic project of showing and sonifying invisible realities 
of electromagnetism. A selection of wonderful images of known and lesser-known works accompanied his 
presentation, tracing history into the present moment, and bringing home the unique situation to discuss 
seminal moments in art history with those that were present there. 

In the evening, the Trädgården started filling up with young club-goers, and it became time for our 
evening program. We started with a brainwave music performance by Jean-Louis Huhta, Per Huttner and 
Robert Oostenveld. Their piece ’an-atomic’ had been performed and fine-tuned several times over the last 
year, and their experience showed. Robert introduced and shortly explained how he would record Per’s EEG 
and used his alpha power to create control signals for both Jean-Louis’ modular synthesizer as well as for a 
video projection. Per was seated with his back to the audience, facing an LED ring that responded to his 
alpha activity, using neurofeedback to help him setting in a receptive state of mental relaxation. Jean-Louis’ 
music slowly faded in, and created an impressive diversity of sonic textures and moods that fit wonderfully 
well with the movie projection. Per’s alpha activity subtly faded between several versions of his movie, 
which were slightly shifted in color or pace, creating an hypnotic responsive background to his meditative 
performance. Per’s combination of images of foliage, abstract images and urban scenes created an poetic 
journey that merged very well with the sonic journey of Jean-Louis’ EEG-controlled composition. 

The brainwave music set was followed by a 
melodic synthesizer and vocal performance by Sonja 
Tofik and Mar-Iena, who provided some breath and 
space, opening up our minds again to the next in- 
tense performance. 

It was then time for Robert Aiki Aubrey Lowe’s 
performance, in which he played his modular synthe- 
sizer bathed in a video projection of stunning color 
schemes, geometric shapes and stroboscopic move- 
ments. His performance was arresting, melting his 
voice, contact microphones, skin conductance mod- 
ules and visual art into a performance that was both 
theatrical and self-absorbed. Every manipulation of 
his modular synthesizer was purposeful and confi- 
dent, expressing an organic connection between sig- 
nals, circuits and himself. While the set was one of 
improvisation - he later told us that things were go- 

Figure 3: Robert Aiki Aubrey Lowe at Trädgården ing another direction than he expected - he was in 
full control of both his mind and his music, drawing 
the rowdy audience in, with his mesmerizing performance. 

The intensity could only go up from there, and was done so by Stockholm-based Kult Dopamin (Didrik 
Goldkuhl), playing his first gig with a relentless set of acid techno, after which the night continued into the 
small hours with a DJ-set by Mar-Ilena and Jean-Louis. 


4 Morning of Saturday June 1st: Trädgården 


We reconvened the next morning for a presentation by David. We expected perhaps too much from our 
audience, and we were only joined by a handful of visitors. This was good fortune for those present, as 
we were able to hold an intimate conversation, triggered by the topics David brought to bear. It would 
be too much to reiterate all that was discussed, so I will only mention some. David explained how he has 
developed his practice by considering ‘intelligent’ agents that interact through, or within, ’musical states’. By 
considering such circular causality between both minds and instruments (and bracketing the elusive concept 
of intelligence), and focussing on the common state in which those can communicate, a more inclusive 
understanding of generative co-evolving music and the musical processes can be gained. This perspective 
was shared by all musicians that were present. For example, Robert Aiki Aubrey Lowe explained how 
both in the brain and in the modular synthesizer, electrical signals are routed and exchanged in a way that 
boundaries between them can become transparent. Such an ’organic’ and communicative relationship is 


part of his musical personal practice, and was apparent in his powerful performance the previous evening. 
Similarly, in Atau Tanaka’s practice, electrical signals from his brain are translated to his muscles, which 
in turn are transformed into sound - creating a circularity in which the instrument is imbued with an 
intelligence that for all intents and purposes has a mind (body?, music?) of its own. Triggered by a question 
of Severine Samson, we also discussed the concept of improvisation and composition in the light of electronic 
music. The musicians all agreed that this distinction only makes sense in the western classical tradition 
where compositions are created separately by composers to be played by performers. In other times, and 
cultures, such distinction falls apart. Composition-improvisation should perhaps be considered as the way 
an expert musician listens and responds with his/her full cultural history and personal training. Rather than 
considering composition and improvisation as separate processes, (neuro)science of music might therefor do 
well to leave such an artificial distinctions behind and rather explore the decision-making processes that 
musicians make at every step of their performance. We all understood that such an integrated perspective 
is a very complex one, however, especially since it has to involve the unique individual background and 
experience of the musician. 

David also presented an historical overview of his work, starting with his early experiments and ending 
with his recent compositions of brainwave music Hobscotch, an opera that occurred in 24 limousines, and in 
which brain activity of visitors controlled singing responses to increasingly personal questions *. Several early 
performances by David were unknown to us, and it brought home how much David had not only explored 
brain activity in music, but how he has engaged the full spectrum of visual art and performance. As an 
example, one early installation involved a double mirror placed between two participants. Each participant’s 
EEG was recorded, with alpha controlling the brightness of the light at the opposite site of the double mirror. 
Due to the effect of light on a double-sided mirror, depending on their attentional state participants were 
either looking at themselves (more light on their side), or at the other (more light at the other side). It is a 
beautiful and elegant piece that stimulates thinking on empathy and brain synchronization in a way that is 
still relevant, not just in art, but certainly for the neuroscience as well. 

Although we had planned some time to have the audience members experience the EEGsynth themselves, 
we decided to continue the conversation and regroup later that day for the final series of performances. 


5 Evening of Saturday June Ist: R1 


The evening started by a brainwave music perfor- 
mance with Samon Takahasi on EEG, and me on 
modular synthesizer, using the EEGsynth to mod- 
ulate the patches with Samon’s alpha activity. The 
enormous reverberations of the reactor hall allowed 
a sparse use of sounds and percussion. I therefor 
used a combination of a Verbos complex oscilla- 
tor, several filters, a flanger, low-pass gate, bucket 
brigade delay, and stereo tube overdrive, to create a 
drone with minimal percussion, and in which I could 
‘dial in’ the modulation by Samon’s alpha activity. 
Like always, the patch quickly became an improvisa- 
tion, and sounds appeared unplanned and seemingly 
outside of my control, only to be nurtured once their 
source was found. I much enjoy those unexpected 
movements that occur when a ’sweet spot’ is, and 
felt happy with the result. 
Figure 4: Ludvig Elblaus & Gerhard Eckel at R1 Second was a performance by Ludvig Elblaus 
and Gerhard Eckel, who had installed four micro- 
phones throughout the hall and reactor core, requiring over two hundred meters of microphone cable. During 
their performance, Ludvig and Gerhard were standing behind two midi controllers, which were connected 
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to two resonating filter banks running on Super Collider*+. They created and sculpted the sound using only 
two speakers and their reverberations and the sound source. Their installation was set on a stainless steel 
trolley, with just the half-open laptop and two controllers, while the sound moved around them as if they 
were weather-controllers conjuring up a storm, or a pair of ghostbusters exorcising an haunted house. People 
slowly moved around, climbed the balconies and explored the living sonic space that they created. Sounds 
and their complex echos moved through and around you, in and out of experience, dilating time and space, 
disorienting then calming once you allowed yourself to settle int its sonic bath. 

It was then time for a longer break at the heart-beat bar, while we set up for an improvisational jam with 
David Roosenboom, Atau Tanaka, Ludvig Elblaus, Gerhard Eckel, Jean-Louis Huhta, Per Huttner, Samon 
Takahasi, and me. This time, Atau used our hardware to connect to his patches build in MAX/MSP. David 
his own EEG recordings to control the musical patches he programmed in Reactor, and played his violin as 
well, while Ludvig and Gerhard used their installation to feed off the sounds we created. Jean-Louis and Per 
started, David followed, after which we took our turns and played with each other, creating a large collage 
of sounds, integrating three EE recordings, EMG recordings, using both software and modular synthesizers. 
At one point I saw and heard Atau move perfectly synchronous with a rhythm of high-pitched glissandi I was 
making with a resonating vactrol filter. For a little I was confused, not knowing whether it was Atau or me 
who were creating the sounds, trying to listen and watch while tracing my patch chords, until I could only 
come up with the conclusion that Atau was moving and playing himself synchronously with my fluttering 
notes, so precisely that I could not tell who came first and who followed. It was then clear how much Atau 
was in control of his body-instrument. The performance build up to a load crescendo, then build down until 
only sounds triggered by David’s brain remained, then faded out. It was a blast, and if the reaction of the 
audience was anything to go by, quite a unique experience. 


6 Sunday afternoon, June 2nd: EMS 


We closed Transformation at the Elektronmusik 
Studion (EMS) for a private meeting. We had the 
fortune to have David explain to us the history and 
design of the Buchla 300, of which EMS is one of 
the few owners of the few existing. 

Discussing all our plans for the future in detail 
would be out of the scope of this report. However, 
we would like to share one perspective. We live in 
very different time today, compared to the early ex- 
periments in brain-based art and performances. On 
the one hand, the possibility of electronic sound syn- 
thesis has become available for anyone with a small 
budget to spend on consumer hardware synthesizers 
or virtual synthesizers. Programming and an un- 
derstanding of electronics have become unnecessary. 
Similarly, EEG has moved from the laboratories into 
the real-world with the availability of cheap con- 
sumer systems which, while often making question- 
able claims and questionable recordings, do allow 
home experimentation. These technological changes 
stand in stark contrast with their use in art, which 
does not seem to have evolved that much. It would 
not (only) be a matter of friendly affectation to say 
that David Rosenboom has been carrying the torch, 
consistently exploring new ideas, both in theory and 
practice, going beyond what most of us have even 


Figure 5: The Buchla synthesizer at EMS 
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5A Ganglion board from the OpenBCI project, fitted in a 3D printed housing designed by Robert Oostenveld 


attempted. Two challenges might be responsible for this slow adoption of EEG in art. Firstly, while their 
use has become easier, understanding EEG, and to some degree basic neuroscience, is still required to make 
productive use of it. Secondly, the actual interfacing with EEG still requires programming against devices, 
if not of EEG analysis itself. In fact, most BC(M)I devices face similar challenges and similar solutions can 
be adopted. In other words, there is a need for sharing solutions for BCI, especially in the field of art, where 
artist might be less independent dealing with engineering problems. The EEGsynth offers such a solution, 
with a well-documented flexible architecture that can be easily patched for purpose of the artist, requiring 
only the editing of text-files, and a basic understanding of the command-line. Importantly, patches can be 
fully documented, saved and distributed for reproduction, offering examples and ideas for further exploration 
by others. By using Python as a programming language, and GitHub for version control, everyone is (at 
least in principle) able to read the code and contribute with additions and improvements. This is further 
simplified by using a modular architecture in which the BC(M)I loop is constructed of individual parts that 
can be adapted to one’s own purpose, taken from an increasing collections of modules that deal with any- 
thing from interfacing with different EEG hardware and musical devices, to the analysis of muscle, EEG or 
heart-beat or eye-blinks, or dealing with the calibration issues and recording and playback of original data 
or it’s analysis. We have been able to use the EEGsynth in a wide variety of performances over the last five 
years, and several independent projects have used the EEGsynth. In this way, the artist could spend his/her 
time developing the concept and experience of their performance, documenting the performance to the point 
that it could be re-enacted in the future. One related plan for the future is to create several re-enactments of 
seminal early EEG-based works as an ode those pioneers, and to guarantee that the ideas and lessons drawn 
from those early works are shared, developed further. 
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